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This paper establishes new restricted isometry conditions for compressed sensing and 

affine rank minimization. It is shown for compressed sensing that S k + 9 kk < 1 guarantees 
the exact recovery of all k sparse signals in the noiseless case through the constrained i\ 

m , 

l/"") \ minimization. Furthermore, the upper bound I is sharp in the sense that for any e > 0, 

the condition b k + 9 kk < 1 + e is not sufficient to guarantee such exact recovery using any 

■^r . 

recovery method. Similarly, for affine rank minimization, if S^ 4 + d^ 4 , < 1 then all matrices 

m 

with rank at most r can be reconstructed exactly in the noiseless case via the constrained 

nuclear norm minimization; and for any e > 0, S^ 4 + O^ 4 , < 1 + e does not ensure such 

1^^ i exact recovery using any method. Moreover, in the noisy case the conditions S k + & kk < 1 

H ■ 

and S^ 4 + Q^\ < 1 are also sufficient for the stable recovery of sparse signals and low-rank 

matrices respectively. Applications and extensions are also discussed. 
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1 Introduction 

Compressed sensing has received much recent attention in signal processing, applied mathematics 
and statistics. A closely related problem is affine rank minimization. The central goal in these 
problems is to accurately reconstruct a high dimensional object of a certain special structure, 
namely a sparse signal in compressed sensing and a low-rank matrix in affine rank minimization, 
through a small number of linear measurements. Interesting applications of compressed sensing 
and affine rank minimization include coding theory [TJ [13], magnetic resonance imaging [22], 
signal acquisition [161 EH] > radar system [H [211 E2] and image compression [271 ED] ■ 

In compressed sensing, one wishes to recover a signal /3 G MP based on (A, y) where 

y = Ap + z. (1) 

Here A G M. nxp is a given sensing matrix and z G W 1 is the measurement error. In affine rank 
minimization, one observes 

y = M{X) + z (2) 

where M : M. mxn — > M. q is a known linear map, X G W nxn is an unknown matrix, and z£l' 
is an error vector. The goal is to reconstruct X based on y and the linear map Ai. In these 
problems, the dimension is typically much larger than the number of measurements, i.e., p ^> n 
and min(m, n) ^> q. A rather remarkable fact is that, when the signal /3 is sparse and the matrix 
X has low rank, they can be reconstructed exactly in the noiseless case and stably in the noisy 
case using computational efficient algorithms, provided that the sensing matrix A and the linear 
map M satisfy certain restricted orthogonality conditions. 

For the reconstruction of /3 and X, the most intuitive approach is to find the sparsest signal 
or the lowest-rank matrix in the feasible set of possible solutions, i.e., 

minimize ||/3||o, subject to A/3 — y G B 
minimize rank(X), subject to A4(X) — y G B 

where ||/3||o denote the £o norm of (3, which is defined to be the number of nonzero coordinates, 
and B is a bounded set determined by the error structure. However, it is well-known that 
such methods are NP-hard and thus computationally infeasible in the high dimensional settings. 



Convex relaxations of these methods have been proposed and studied in the literature. Candes 
and Tao [13] introduced an t\ minimization method for the sparse signal recovery and Recht, et 
al |27] proposed a nuclear norm minimization method for the matrix reconstruction, 

{Pi) ft = argmin{||/3||i subject to A/3 - y e B} , (3) 

$ 

(Pj) X* = argmin{||X||* subject to M{X) - y G B} , (4) 

x 

where ||X||* is the nuclear norm of X which is defined to be the sum of all singular values of 
X. Here B = {0} in the noiseless case and B is the feasible set of the error vector z when z is 
bounded. These methods have been shown to be effective for the recovery of sparse signals and 
low-rank matrices in a range of settings. See, e.g., [131 O DS 123 E5J- 

One of the most commonly used frameworks for compressed sensing is the Restricted Isometry 
Property (RIP) introduced in [13]. The RIP framework was later extended to the affine rank 
minimization problem by Recht et al in [27] • A vector is said to be k-sparse if |supp(t>)| < k, 
where supp(u) = {i : Vi 7^ 0} is the support of v. We shall use the phrase "r-rank matrices" to 
refer to matrices of rank at most r. For matrices X = (xij) £ W mxn , and Y = (yij) £ W mxn , 
define the inner product of X and Y as (X, Y) = trace(X r Y) = ^™=i S?=i x ijVij- The norm 



associated with this inner product is the Frobenius norm, ||^"||f = yi^iX) = yY^Li S?=i 
The following definitions are given by \12>\ [271 123] . 
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Definition 1.1. Let A € M nxp and let 1 < k,k\,k2 < p be integers. The restricted isometry 
constant (RIC) of order k is defined to be the smallest non-negative number 5~£ such that 

(i - s£)Ml < Wg < (1 + 6£)\\pg (5) 

for all /c-sparse vectors /3. The restricted orthogonality constant (ROC) of order (fci,/^) is 
defined to be the smallest non-negative number 9^ k such that 

^A^Afo^KOl^hWfoh (6) 

for all fei-sparse vector (3\ and /^-sparse vector ,$2 with disjoint supports. 

Similarly, let M : W nxn — > MP be a linear map and let 1 < r,r±,r2 < min(m, n) be integers. 
The restricted isometry constant (RIC) of order r is defined to be the smallest non-negative 
number 8^ such that 

(1 - ^)\\Xf F < \\M(X)\\l < (1 + Oll^lll (7) 



for all m x n matrix X of rank at most r. The restricted orthogonality constant (ROC) of order 
(j"i,r 2 ) is defined to be the smallest non-negative number 0^ r such that 

\(M(X 1 ),M(X 2 ))\ < < )fc2 ||X 1 || F ||X 2 || J , (8) 

for all matrices X\ and X2 which have rank at most r\ and r 2 respectively, and satisfy X{X 2 = 
and X x Xl = 0. 

In addition to RIP, another widely used criterion is the mutual incoherence property (MIP) 
defined in terms of fi = max^- K^j, A,)|. See, for example, [191 E]- The MIP is a special case 
of the restricted orthogonal property as \x = Q\^\ when the columns of A are normalized. 

Roughly speaking, the RIC 8 k and ROC 9 k k measure how far subsets of cardinality k of 
columns of A are to an orthonormal system. It is obvious that 8k and 9k lt k 2 are increasing in 
each of their indices. It is noteworthy that our definition of ROC in the matrix case is different 
from the one given in 



Sufficient conditions in terms of the RIC and ROC for the exact recovery of /c-sparse signals 
in the noiseless case include 8$ + 9^ k + 9^ 2k < 1 [13]; 8% h + Q k2k < 1 [33]; 5^ 5k + 0^ 15k < 1 [5], 

S i.25k + ^1.25*; < l 0' and e ii < 2iki when S i = ° C3 123 U\ ■ Sufficient conditions for the 
exact recovery of r-rank matrices include 8 2 r+ar + -j^9 2r+ar ^ r < 1 where 2a < /3 < 4a [23]. It 
is however unclear if any of these conditions can be further improved. 

In this paper we establish more relaxed RIP conditions for sparse signal and low-rank matrix 
recovery. More specifically, we show that the condition 

s£ + e£ tk <i (9) 

guarantees the exact recovery of all A>sparse signals in the noiseless case via the constrained l\ 
minimization ([3]) with B = {0}. Furthermore, we show that the constant 1 in ([9]) is sharp in the 
sense that for any e > 0, the condition 5 k + 6 kk < 1 + e is not sufficient to guarantee such exact 
recovery using any method. Similarly it is shown that the condition 

5^ + 0# < 1 (10) 

is sufficient for the exact reconstruction of all r-rank matrices in the noiseless case through 
the constrained nuclear norm minimization §%§ with B = {0}, and that for any e > 0, the 



condition 5^ + 9^. < 1 + e is not sufficient to guarantee such exact recovery using any method. 
Moreover, in the noisy case the conditions (J9j) and (|10p also guarantee the stable recovery of 
sparse signals and low-rank matrices respectively. In addition to the sufficient conditions ([9]) 
and (|10p . extensions to the more general RIP conditions are also considered. 



The new RIP conditions are weaker than the known RIP conditions in the literature. The 
techniques and results developed in the present paper have a number of applications in signal 
processing, including the design of compressed sensing matrices, signal acquisition, and analysis 
of compressed sensing based radar system. We discuss these applications in Section SJ 

The rest of the paper is organized as follows. In Section [21 we first introduce the basic 
notations and definitions and then present the main results for both sparse signal recovery and 
low-rank matrix recovery. Extensions of the results 8£ + 9 kk < 1 and 5^ + 9y\, < 1 to the more 
general RIP conditions are also considered. Section [3] discusses the relationship between our 
results and other known RIP conditions. Section H] illustrates some applications of the results 
in signal processing. The proofs of the main results are given in Section [5j 

2 New RIP Conditions 

We present the main results in this section. It will be first shown that the conditions 8 k +9 kk < 1 
and 5p + 9^, < 1 are sharp for the exact recovery in the noiseless case and stable recovery in 
the noisy case. The more general RIP conditions will be considered at the end of this section. 

Let us begin with basic notation. For v S MP, f ma x(fc) is defined as the vector v with all but 
the largest k entries in absolute value set to zero, and f_ max (A;) = v — u max (fc)- For a matrix 
X G ig mxn (without loss of generality, assume that m < n) with the singular value decomposition 
X = YITLi a i u i v T where the singular values ai are in descending order cti > 02 > • • • > a m > 0, 
we define ^ max (r) = J2l=i a i u i v J an d X_ max ( r ) = X — X max ^ . We should also note that the 
nuclear norm || • ||* of a matrix equals the sum of the singular values, and the spectral norm || • || 
of a matrix equals its largest singular value. Their roles are similar to those of l\ norm and t^ 
norm in the vector case, respectively. For a linear operator M : W nxn — > M q , we denote its dual 
operator by M* : W ->■ M. mxn . 



It follows from [25] that the results for the low-rank matrix recovery are parallel to those 
for the sparse signal recovery. So we shall present the results for the two problems together in 
this section. The following theorem shows that the conditions Q and (|10p guarantee the exact 
recovery of all fe-sparse signals and r-rank matrices through the constrained i\ minimization and 
constrained nuclear norm minimization respectively. 

Theorem 2.1. Let /3 £l p be a k-sparse vector and y = A/3. If 8 k +9 kk < 1, then J3 = f3, where 
/3 is the minimizer of ([3]) with B = {0}. Similarly, let X be an r-rank matrix and y = Ai(X). 
If 5^ + 6^1 < 1, then X* = X, where X* is the minimizer of @ with B = {0}. 

We now turn to the noisy case. Although our main focus is on the recovery of sparse signals 
and low-rank matrices, we shall state the results for general signals and matrices that are not 
necessarily sparse or low-rank. 

We consider two bounded noise settings: ||z||2 < e, and H^^^Hoo < e (signal case) and 
\\M* (z) || < e (matrix case). The case of Gaussian noise, which is of significant interest in 
statistics, can be essentially reduced to the bounded noise case. See, for example, Section 4 in 
[U] for more discussions. In the theorems below, we shall write 5 for 5 k and 8^ and write 9 for 
9 kk and 9 kk . We first consider the case where the £2 norm of the error vector z is bounded. 

Theorem 2.2. Consider the signal recovery model ([1]) with \\z\\2 < £• Let f3 be the minimizer 
of ([3]) with B = {z G R n : ||z|| 2 < ??} for some 7] > e. If 5 k + 9 kk < 1 for some k > 1, then 

Similarly, consider the matrix recovery model (J2]) with \\z\\2 < e. Let X* be the minimizer of (JJ|) 
with B = {z G M. q : \\z\\2 < n} for some n > e. If 5^ + 9^j, < 1 for some r > 1, then 



II y Y\\ < y^+^ f , I ^ I 2 W X -^(r)\U / V29 \ 

We now consider the case where the error vector z is in a polytope defined by ||A z||oo < e 
and ||A4*(«)|| < e. This case is motivated by the Dantzig Selector method considered in [TJ] for 
the Gaussian noise case. 



Theorem 2.3. Consider the signal recovery model ([T]) with ||A «||oo <• e - -^ei/3 &e the minimizer 
of © ?mtt S = {z G l n : H^zH^ < 77} /or some rj > e. If 5 k + 9 kk < 1 for some k>l, then 

Similarly, suppose we have the signal and matrix recovery model ([2]) with \\A4*(z)\\ < e. Let /3, 
X* be the minimizer of (@J wi/i 6 = {z e I' : ||A^*(z)|| < rf\ for some rj > e. If 6^ + 6^. < 1 
/or some r > 1, then 



Theorems EZQ [2J2 and O shows that the conditions 5^ + 9^ k < 1 and o"^ + &Q < 1 
are respectively sufficient for the exact and stable reconstruction of sparse signals and low-rank 
matrices via the constrained t\ minimization and nuclear norm minimization. The following 
theorem shows that the upper bound 1 in these conditions is in fact sharp. 

Theorem 2.4. Let 1 < k < p/2. There exists a sensing matrix A € R nxp such that 5u+0 kk = 1 
and for some k-sparse signals u,v £ W with u ^ v, Au = Av. Consequently, there does not 
exist any method that can exactly recover all k-sparse signals (3 based on (A,y) with y = A/3. 

Let 1 < r < min(m,n)/2. There exists a linear map Ai such that 5^ + 6^. = 1 and for 
some matrices U, V € ]R mxn with rank(C/), rank(V) < r, and A4(U) = M(V). Therefore, it is 
impossible for any method to recover all r-rank matrices exactly based on (A4,y) with y = M(X). 

Remark 2.1. Theorem 12.41 implies that for any e > 0, 5 k + 9 kk < 1 + e fails to guarantee the 
exact recovery of all fc-sparse signals. These results immediately show that for any e > 0, the 
condition 5 k + 9 kk < 1 + e or 5^ + O^j. < 1 + e is not sufficient to ensure in the noisy case stably 
recovery of all fe-sparse signals and all r-rank matrices. 

Remark 2.2. The results on the bounded noise case can be applied to immediately yield the 
corresponding results for the Gaussian noise case by using the same argument as in [5j [6]. We 
illustrate this point for the signal recovery. Suppose z ~ J\f n (0,o~ 2 ) in ([1]). Define B DS = {z : 
\\*& «||oo < o"\/21ogp} and B 2 = {z : \\z\\2 < oy n + 2y/n log n}. Then, with probability at 
least 1 — , , the Dantzig selector /3 given by ([3]) with B = B satisfies 



and the £2 constraint minimizer J3 2 denned in ([3]) with B = B 2 satisfies 



WRh fill <T Vgl±jj / , " rn 1 2 H/ 3 -max(fc)lll V29 \ 

with probability at least 1 — 1/n. We refer readers to [5j [6] for further details. 



Extensions to More General RIP Conditions 

We have shown that the conditions 5£ + 9 kk < 1 and 8^ + 9^j. < 1 are sufficient respectively 
for sparse signal recovery and for low-rank matrix recovery. The same techniques can be used 
to extend the results to a more general form, 



rA /~t nA ' — ^ ' — ^ 



S a + C a ,b,k e a,b < 1> where C ai & :fe = max < , \/ — — - <a<k, (17) 



^ + C aAr e%i < 1, where C a , 6 , r = max J %=^, J— - I , 1 < o < r. (18) 

Theorem 2.5. In t/ie noiseless case, Theorem \2. 1\ holds with the conditions 5 k + 9 kk < 1 and 
5^ + 6^. < 1 replaced by (fT7|) and (|TH|) respectively. 



In the noisy case, we have the following two theorems parallel to Theorems 12.21 and [2? 

Theorem 2.6. Consider the signal recovery model ([1]) wrat/i ||z||2 < e. Lei /3 be the minimizer 
of Q with 6 = {2 £ 1" : ||z||2 < 77} for some rj > e. If 5^ + C a ^^9^ b < 1 for some positive 
integers a and b with 1 < a < k, then 

W - ph - i-s-c a ^ + ^ ) + 2 ^-^ ^ (19) 

Similarly, consider the matrix recovery model ([2]) urai/i ||z||2 < e. £e£ X* &e i/ie minimizer of @ 
wi/i B = {z £ l' : ||z||2 < n} for some r\ > e. If 5^ + Ca,6,r^o6 ^ ^ -^ or some positive integers 
a and b with 1 < a < r, t/ien 



J2(l + S)rla , , „ / y/2rC abr 9 1 \ , , 

1 - (5 - G aib)T .0 Wl-d-C aj6)r .0)(2r-o) V r / 

Theorem 2.7. Consider the signal recovery model ([T]) wi/i ||vl 2||oo ^ e - -£e£/3 &e i/ie minimizer 
of ([3]) wii/i £? = {z € M n : jj^zHoo < 77} /or some t] > e. If 6^ + C a ^,k^ab < ^ / or some positive 



integers a and b with 1 < a < k, then 



I, a ,,, V2fe ( , ., „ ||fl I, / V2kC aM 6 1 \ 

^-^- l-J-C.,^ ^*^ (21) 

Similarly, suppose we have the signal and matrix recovery model (|2|) with \\A4*(z)\\ < e. Let (3, 
X* be the minimizer of © with B = {zeR q : ||.M*(z)|| < r/} /or some 77 > e. If 5^ +C aAr 9^ b < 
1 /or some integers a and b with 1 < a < r, ift-en 

||X* -^||f < ^ 7 ^ n( e + r ?) + 2 ll^-max(r)||* Tj 7 ^ ^7a7^ V + "P ' ( 22 ) 

1 - 5 - C a)6)7 .0 v ; V (1 - 6 - C a}h>r e)[2r - a) Jr J 

The next theorem shows that the upper bound 1 in the conditions 5^ + C a M,kS^ b < 1 and 
o"„ + Ca : b, r &a,b < 1 cannot be further improved. 

Theorem 2.8. Let 1 < k < p/2, 1 < a < fc, and b > 1. Lei C a &fc 6e defined as (fT7|) . TTien 
i/iere exists a sensing matrix A G R nxp suc/i t/iai <5„ + C a fi,k@ab = 1 anc ^ / or some k-sparse 
signals u,v £M P with u^v, Au = Av. Consequently, there does not exist any method that can 
exactly recover all k-sparse signals (3 based on (A, y) with y = Af3. 

Similarly, let 1 < r < min(m,ra)/2, 1 < a < k and b > 1. Let C a ,b,r be defined as (|18p . Then 
there exists a linear map M such that 6"„ + C a% b,rQa\, = 1 and for some matrices U, V S j^ TOXn 
mt/i rank(?7), rank(V) < r, and A4(U) = M(V). Consequently, it is impossible for any method 
to exactly recover all r-rank matrices based on (M,y) with y = Ai(X). 

Same as Theorem 12.41 Theorem 12.81 implies that in the noisy case stably recovery of all 
fc-sparse signals and all r-rank matrices cannot be guaranteed by <5„ + C a ^,kd^ b < 1 + e or 
5^ + Ca,6, r 0# < 1 + e for any e > 0. 

Remark 2.3. We established the more general RIP conditions 5^ + C a ,b,r^ab ^ ^ an< ^ ^a* + 
C a ,b,r^ab r < 1- For fixed a, among these conditions, the one with b = 2k — a or b = 2r — a is 
the weakest. We shall illustrate this for the signal case. By Lemma [ 



0~a + C a ,2k-a,k^a,2k-a < 0~a + C a ,2k-a,k\ — ; — 77 r,, 7#a,min{&,2fc-a} 

1/ mmjo, Ik — a) l J 



2k -a / 2A;-a 

— "a + A/ • W — : — r , „, VVa,min{b,2k-a} 

V a y mmjo, 2k — a} 
Hence, for all 6 > 1, 5 a + C aybik 9 a)b < 1 implies <5 a + C a ^k-a,k^a,2k-a < 1- 



3 Relationship to Other Restricted Isometry Conditions 

In the last section, we have established the sufficient conditions 8 k + 9 kk < 1 and 8^ + 9^. < 1 
for the exact recovery in the noiseless case and stable recovery in the noisy case. We discuss in 
this section the relationships between these conditions and other restricted isometry conditions 
introduced in the literature. 

By the simple fact that for k\ < k^ and k[ < k' 2 , 8 k — ^t an< ^ ®k k' — ®t k' > ^ * s eas y *° see 
that the condition 6£+0£ k < 1 is weaker than 8^+9^ k +9^ 2k < 1, 8^ k +9^ kk < 1, &tbk+ e i.sk,k < 
1 and ^i 25^+^1 25fc k < -*-' w hich were mentioned in the introduction. Note that setting a = b = 1 
in the condition 8 a + C aj t, t k9 a ,b < 1 yields a sufficient condition 8^ + (2k — l)#i,i < 1 which is 
more general than the MIP condition 9\^ < 2k _ l wnen ^i = given in [19] and |20j for the 
noiseless case and [7] for the noisy case. 

There are also several sufficient conditions in the literature that are based on the RIC 8 alone, 
such as 5£ k + 35f k < 2 [TO], 8& < V2-1 [33]; 5 2k < 0.472 [g]; 8$ < 0.307 0; 8 2k < 0.493 [26] and 
8 k < 1/3 and 8 2k < 1/2 [9]. For the matrix recovery, sufficient conditions include 8^: < y/2 — 1 
[151; <*&? < °- 607 > &£ < °- 558 . and 5 3? < °- 4721 E3]; 6$ < 0.4931 and <5^ < 0.307 [EE], 
and (5^ < 1/3 and 8^ < 1/2 [9j. In particular, Cai and Zhang [9] showed that 8 k < 1/3 and 
<5^ < 1/3 are sharp RIP conditions for the exact recovery. It is interesting to compare these 
results on 8$, Sf k , 8^, and 8g with 8% + 9^ k < 1 and S* 4 + &%. < 1. 

The following lemma provides a bound for the ROC 9 in terms of the RIC 8 and can be used 
to compare different RIP conditions. 

Lemma 3.1. Let A G R nxp . Then we have 

. 2(5i?, when k is even, k > 2; 

eik<{ 2 , . " (23) 

, 2 8 k , when k is odd, k > 3. 

In addition, both coefficients, 2 in the even case and , , 2 in the odd case, cannot be further 
improved. 



Similarly, in the matrix case, for a linear map M. : 



«mxn 



2& V! , when r is even, r > 2; 

<{ (24) 

/ I ^r i when r is odd, r > 3. 

10 



In addition, the coefficient 2 in the even case cannot be further improved. 

With Lemma 13-H we can naturally obtain the following result which shows that the condi- 
tions 8 k + kk < 1 and 8^ + 6^}. < 1 are mostly weaker than the RIP conditions 8 k < 1/3 and 
8^ < 1/3 respectively. 

Proposition 3.1. If S k < 1/3 for some integer k>2, then 



$k + ®k k < Ij when k is even; 

8b + Ou l. < — I ; ~ 1 H — T77 , when k is odd. 

' 3 3\/k 2 - 1 3A: 2 

Similarly in the matrix case, if 8^ < 1/3 for some integer r > 2, then 

8^ + ^rr < 1 ) w/ien fc is even; 

1 2r 1 

o"^ + 0\r„ < — | wH o , lo/ien A; is odd. 

' 3 3Vr 2 - 1 3r 2 



(25) 



(26) 



Sufficient conditions in terms of 8A. and o - ^ are also commonly used in the literature. To the 
best of our knowledge, the weakest bounds on 5%^ and 5$ for the exact recovery are 5%^ < 1/2 
and 8^ < 1/2 given by Cai and Zhang [Sj. It is easy to see that the conditions 8 k +9 kk < 1 and 
8-}r + 9^f\. < 1 given in the present paper are strictly weaker than these conditions respectively. 

Proposition 3.2. If 8^ < 1/2 for some integer k > 1, then8 k +9 kk < 1. Similarly, if 8$ < 1/2 

/or some integer r > 1, £/ien o"^ + #,^ < 1. 

This is an immediate consequence of the results given in Section [2] and the following lemma 
given in |15j . 



Lemma 3.2. Suppose A 6 M. nxp and M is a linear map from M. mxn to M. q , then 

0£ >k <8i k , 0%<5%. (27) 

4 Applications 

As mentioned earlier, compressed sensing and affine rank minimization have a wide range of 
applications. The techniques and results developed in this paper naturally have a number of 

11 



applications in signal processing, including the design of compressed sensing matrices, signal 
acquisition, and analysis of compressed sensing based radar system. We discuss some of these 
applications in this section. 

An important problem in compressed sensing is the design of sensing matrices that guarantee 
the exact recovery in the noiseless case and stable recovery in the noisy case. Different types of 
matrices have been shown to satisfy the previously known sufficient RIP or MIP conditions with 
high probability. Examples include i.i.d. Gaussian matrices [131 114] . general random matrix 
satisfying concentration inequality [3], Toeplitz-structured matrices [2], structurally random 
matrices [T7] and the matrices from transmission waveform optimization [32] . These matrices 
are thus provably suitable for compressed sensing. A direct consequence of the weaker RIP 
condition obtained in this paper is that a smaller number of measurements are required to 
guarantee the exact or stable recovery of sparse signals. 

Take for example i.i.d. Gaussian or Bernoulli random matrices. Theorem 5.2 in [3] shows 
that if a random sensing matrix A = (<%•) & M nxp satisfies 



iid K r/r\ t / \ iid 

dij ~ A/(0,l/n), or ajj ~ < 



1/y/n w.p. 1/2 
-l/y/n w.p. 1/2 



or dij ~ < 



i/3/n w.p. 1/6 
w.p. 1/2 



-y/3/n w.p. 1/6 
then for any positive integer m < n and < t < 1, the RIC 5^ of the matrix A satisfies 

It is helpful to compare the condition 5 k + kk < 1 in terms of these random sensing matrices 
to the best known RIP conditions in the literature: 5k < 1/3 and 82k < 1/2 [9]- Suppose for 
some given < e < 1 one wishes the sensing matrix A to satisfy the RIP condition 5~£ < 1/3 or 
^2k ^ 1/2 with probability at least 1 — e. Then, based on (|28p . for given k and p the number of 
measurements n must satisfy respectively 

log(e/2)" 



n > 162 [k(log(p/k) + 4.6) - log(e/2)] and n > 153.6 



k(log(p/k) + 3.5) 



2 

On the other hand, it is easy to see that 5^ + kk < 1 is implied by 5 k + 8% k < 1 which is in 
turn implied by the condition 5 k < 0.4 and <5^, < 0.6. Note that for given k and p, n> n\ with 

m = 115.4 [k(log(p/k) + 4.4) - log(e/4)] 

12 



guarantees 8 k < 0.4 with probability at least 1 — e/2, and n > n^ with 

log(e/4)" 



n 2 = 111.1 



k(\og(p/k) + 3.3) 



ensures 5^. < 0.6 with probability at least 1 — e/2. Hence, 8£ + 8^. < 1 holds with probability 
at least 1 — e if the number of measurements n satisfies 

n > max{m,n2}. (29) 

Therefore, for large & and p, the required number of measurements to ensure 8 k + 9 kk < 1 is 
less than 71.2% (115.4/162) and 75.1% (115.4/153.6) of the corresponding required number of 
measurements to ensure 8 k < 1/3 and 8^ k < 1/2, respectively. 

The results given in this paper can also be used for certain theoretical analysis in signal 
processing. One example is the signal acquisition problem studied in [16] . Davenport et al [16] 
considered acquiring a finite window of a band-limited signal x(t) given by 

p-l 

x(t) = *(a) =5^a t ^(t), 

i=o 

where ifij(t) = e t2 ' K ^ t (i is the imaginary unit) are the Fourier basis functions, and a = [a^, ai, ■ ■ ■ , Op—i] 
is k sparse. Suppose the measurements yi, • • • ,y n are acquired as 

p— l p— l p— l 

%• = {4>j{^)^{t)) + z = (4>j(t),^2aiTpi(t)} + z = ^2ai((pj(t),ijji(t)) + z = ^rjiai + z 

1=0 1=0 1=0 

where z is measurement error. Then it can be written as 

y = Ra + z, (30) 

which is exactly ([1]). When R = (r^-) with nj i.i.d. Gaussian or Bernoulli, as discussed above, 
the measurement matrix R satisfies the RIP condition of order k or 2k with high probability 
provided that 

n> K klog(p/k), (31) 

in which case stable recovery of the signal x(t) can be achieved through t\ minimization. 



The lower bound of k§ in (13ip is typically computed through simulations [161129] . Our results 
yield a theoretical lower bound for kq, namely kq > 115.4 based on equation ([29H . It is also 
helpful to provide an upper bound for the error of recovery. Suppose that z ~ N n (0, a 2 ) and 
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Condition (|3ip is satisfied. Then (|15|) and (|16p yield that the Dantzig selector and ^2 constraint 
minimizer given in Remark 12.21 satisfy, with high probability, 

\\x(t) DS - x(t)\\ 2 = \\a DS - a\\ 2 < 0^7^^ + C 2 i |a - max(fc)l11 



Vk 

x{tf 2 -x(t)\\ 2 = \\a l2 -a\\ 2 <C 3 aV^ + C2 ' 



^ _^miL- IIA«a _„il ^<T,^/^-L/T. l|a - max Wlll 



where Ci,C 2 , C3 are constants specified in Remark 



In addition, the results obtained in this paper are also useful in the analysis of compressed 
sensing based radar system [I]. Suppose the object of interest is represented by u(t) and the 
transmitted radar pulse for detecting the object is st(£). Then the received radar signal is 
s R.(t) = cf sx(t — T)u{r)dt. Baraniuk and Steeghs [4] discretizes this equation and the com- 
pressed sensing based radar model then becomes 

N 

sji(mDA) = c y p(mD — n)u(nA), m = 1, • • ■ , M 

n=l 

which is the same as the compressed sensing model ([1]) in the noiseless case. Whether it is 
possible to recover the signal u(t) with accuracy requires checking the condition on the matrix 
A = (a mn )MxN with a mn = p(mD — n). Weaker RIP condition makes it easier to guarantee the 
recovery of the signal u(t). 

5 Proofs 

We now prove the main results of the paper. Throughout this section, we shall call a vector an 
"indicator vector" if it has only one non-zero entry and the value of this entry is either 1 or — 1. 

We first state and prove a key technical tool used in the proof of the main results. It provides 
a way to estimate the inner product (a,j3) and (X\,X 2 ) by the ROC when only one component 
is sparse or low-rank. 

Lemma 5.1. Let k±,k 2 < p and A > 0. Suppose a,f3 G W have disjoint supports and a is 
k\-sparse. If \\f3\\i < Xk 2 and ||/3||oo < A, then 



\(Aa,Af3)\<e£ lM \\a\\ 2 -Wk 2 . (32) 
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Let ri,r 2 < min{m, n} and A > 0. Suppose X±,X 2 G W nxn satisfy XfX 2 = 0, X\X 2 = 0, and 
rank(Xi) < r\. 7/||X 2 ||* < Ar 2 and \\X 2 \\ < A, £/ien 

\(M(X 1 ),M(X 2 ))\<9^ k2 \\X 2 \\ F .\^r 2 . (33) 

Proof of Lemma 15.11 We first state the following result which characterizes the property of 
X and Y when X T Y = and XY T = 0. The result follows directly from Lemma 2.3 in [27] 
and we thus omit the proof here. 

Lemma 5.2. For X, Y G M. mxn , X T Y = 0, XY T = if and only if there exist orthonormal 
bases {mj G M m : 1 < i < m} and {t>j G R n l < i < n} suc/i i/iat i/ie singular value decompositions 
of X and Y have the form 

X = y^ aiUiV i and Y = \^ huiVi 
where T\ and T 2 are disjoint subsets of {1, • • • , min(m, n)}, cjj, bj > 0. 

We shall only prove Lemma[5j]for the signal case as the proof for the matrix case is essentially 
the same. Suppose ||/3||o = I, then j3 is an /-sparse vector. When I < k 2 , by the definition of 
5£ iM , \(A(a),A({3))\ < ^ lifc2 ||a|| 2 ||/3|| 2 < 0& )fc Ja|| 2 V£A since ||/3||oo < A. Thus © holds for 
l< k 2 . 



Now consider the case I > k 2 . We shall prove by induction. Assume that ([32]) holds for I — 1. 
For /, suppose /3 can be written as X 2 = Ei=i c i u i-> where c\ > c 2 > • • • > q > 0, {ui} l i=1 are 
indicator vectors (defined in the beginning of this section) with different supports. Notice that 

E!=i °i ^ AA; 2 < (I - 1)A, sol€D = {l<i<l-l:Cj + c j+ i H + q < (I - j)X}, which 

means that D is non-empty. We can pick the largest element j G D, which implies 

cj + cj+i -\ h ci < (I - j)X, c j+ i + c j+2 -\ h ci > (I - j - 1)A. (34) 

(It is noteworthy that even if the largest j in D is I — 1, (|34p still holds) . Define 

Cm, j <w <l (35) 



Z^i=j c * 



and 






" 



w 



-I 



Y^ cm + Yl dwUi G RP ' j - l - L ( 36 ) 



Et j / j / j 
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It is easy to check that Y^ w =j 7™ = P, 12i=j c i = { l ~ J) 52i=j d%- % ©, for all j < w <l, 

dw > dj = E H +1 C% ~ [ ^ 1 c J > g^ « ~ g " J " 1)A > 0. 

* - j l-j 1-3 



We also have 

ii7«iii = r J2a+(i- j)d w = r Q2 ^+yi ^ = ^r , i^iii ^ w ™ , Afc2 ' 



and 



d w d w , -, _ r d w , d w {2_, i= j Ci) d w 



h^Hoo = max{ Cl , • • • , Cj-i,^} < max{ A, , — } < , A. 

!,*=,•* 22i=jdi 22i=j d i (« ~ 3)(22i=j d i) 22i=j d i 

The last inequality follows from the first part of (|34p . Finally, since j w is (I — l)-sparse, the 

induction assumption yields that 

i i , 

\{Aa,A(3)\ <Y,Ua,A lw )\ < ^ lifc2 ||a|| 2 ^ -r^ A V^ = ^ llfc2 ll«|| 2 A^ 

w=j w=j £-~/i=j l 

which gives (|32p for /. □ 

Proof of Theorems 12.11 and 12.51 It suffices to prove Theorem 12.51 as Theorem 12. II is a spacial 
case of Theorem 12.51 We first state two lemmas. Lemma l5.3( which characterizes the null space 
properties, is from [28| and [21]. Lemma 15.41 which reveals the relationship between ROC's of 
different orders, is from [6]. 

Lemma 5.3. In the noiseless case, using ([3]) with B = {0} one can recover all k-sparse signals 
/3 if and only if for all h G M(A)\{0}, 

2 l|^max(fc)l|l < INK- 

Similarly in the noiseless case, using (j4]) with B = {0} one can recover all matrices X of 
rank at most r if and only if for all R G Af(M)\{0}, 

*• 1 1 -^max(r) 1 1 * ^ II-*hI* - 

Lemma 5.4. For any \x > 1 and positive integers k\^ki such that \iki is an integer, then 
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As mentioned before, by [25], the results for the sparse signal recovery imply the correspond- 
ing results for the low-rank matrix recovery. So we will only prove the signal case. By Lemma 
15.31 it suffices to show that for all vectors h £ N(A) \ {0}, ||/i max (fc)lli < ll^-max(fc)l|i- 

Suppose there exists h € AA(^4)\{0} such that ||/i ma x(fc)l|i > ll^-max(fc)l|i- Let h = Y^=i c i u ii 
where {cj}^ =1 is a non-negative and non-increasing sequence; {wj}f =1 are indicator vectors (de- 
fined at the beginning of this section) with different supports in W. Then we have ^i=i c * — 

£f =fc+1 Q. HenCe - ll^-max(a)Hoo = C a+1 < ^1 = IIW(.)[|i and 

k p , k k , a * a ^i 

.. .. ^ — *\ ^ — > /C — (1 ^ — > ^ — > /C — (1 ^ — > r£ ^ — > ZrC — (I 

||/i-max(a)||l = }^ ° i+ Z> Ci ^ ~ 7" ,L Ci +L Ci ^ ~ ^~ Z> Ci + ~ Z> Ci = ~ ^~ ll^max(a)| 
i=a+l %=k+l i=l i=l i=l i=l 

We set A = IIVax(a)l11 , k x = a, k 2 = 2k - a, It then follows from Lemma O that 



A .fKT -||i, II llVax(a)||l A 2k - CI 2 



I / A l u \ A { u \\\ ^ nA /rw 1| u " '"max^aj || i ~ A / II 7 II 2 

K^(' i max(a)j ; ^("'-max(a)J)l S ^ a ,2k-a V 2k - a||/l m ax(a) ||2 — i ^ a ,2k-a\J ~ ll^max^lb 

On the other hand, Lemma 15.41 yields 



2k — a „ , 1 2k — a 

1a,2k-a < 



mini 



-TTWi T^a,mm{b,2k-a} < max < \ , 1 > 6 a>b . 

i{b, 2k — a) > L ' J V b 

Hence, 

= \(A(h max{a) ),A(h)}\ >\(A(h ma ^ a) ),A(h max{a) ))\-\(A(h max(a) ),A(h_ max{a) )}\ 



2k — a 

(1 ~ K )ll^max(a)lb ~~ ^a,2k-a\/ ~ ll^max(a) lb 



,_, r/ i \ 2k — a 2k — a nA .... Il2 

(1 -5 a -max<( r=-,\l f ^bJll^max^lb 



<ab 

0--fi a - C a ,b,k8 a ,b)\\hma, x (a)\\2 

which contradicts the fact that h / and 5^ + C a ^.k6ab < 1- □ 

Proof of Theorems l2T2l l2T3l I2T61 and [2771 Again, it suffices to prove Theorems EH and [2771 
We need the following Lemma 15.51 from [9] which provides an inequality between the sums of 
the pth power of two sequences of nonnegative numbers based on the inequality of their sums. 

Lemma 5.5. Suppose m > r, a\ > a 2 > • • • > a m > 0, and YH=i a i ^ S2=r+i °«- Then for all 

m r 
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More generally, suppose A > 0, a\ > 02 > ■ ■ ■ > a rn > 0, and Yl\=i a« + A > Y^Lr+i a «> then for 
all p > 1, 

t-^A^< + A\ (38) 

j=r+l \ / 



We first prove Theorem l2.2i Set h = J3—f3 and R = X*— X. The following inequalities are well 

known, ||/t_ max (fc)||l < ll^max(fc)lll+2||/3-max(/c)l|l an d H-R- max(r) II* < ll-^max(r-) ll*+2||X_ max(r) II*- 

See, e.g., [18] (signal case) and [31] (matrix case). Again, we only prove the signal case. By the 
boundedness of z and the definition of the feasible set for j3, 

\\Ah\\ 2 < \\Ah - y\\ 2 + \\y - A0\\ 2 <e + V . (39) 

On the other hand, suppose h = Yli=i c i u i-> where {cj}^ =1 are non-negative and non-decreasing, 
{uj}^ =1 are indicator vectors with different supports. Then 

m k 

Y^ Ci<Y,Ci + 2||/3_ max(fc) ||i. (40) 

i=k+l i=l 

nence, ||/T._ max ( a )||oo — c a +i S „ — „ S - I gfc^ ana 

k p , k k 

l|fr-max(a)l|l ^Z Ci + ^ Ci - ~ ~ 5Z Ci + X] Ci + 2 H^~ max ( fc ) H 1 



fi Li ^ — A A, ^ ""\ . - if A, ^ II 7 C\\\ n 

< Z^ Ci + _ 2^ Ci + 2 IIP-max(fc)l|l = ||"max(o)l|l + 2 I|P- max(fc) 111- 

i=l i=l 

Now set A = IIVa ^ (Q)l11 + m ~2™a k)h 1 fc i = a ' and k 2 = 2k- a. Lemma O then yields 

l/yt/l. \ Atu \\\ ^ oA [K] NT, 11 / ll"max(a)l|l *\\P- max(fc) II 1 \ 

|(^(Vax(a)),^(/i-max(a)))l < 6 a ,2k- a v2k - a||/l max(o) || 2 ■ I + ^— I • 

On the other hand, 

|(A/l,A/l max(a) )| < ||vl/l||2pVax(a)l|2 < (e + r/)Vl + 5||/l max ( a ) || 2 . (41) 

Now we denote Q a ,2k-a as 0, then 

(e + ry)\/l + 5||/l max ( a )|| 2 > |(A/l, A/l m ax(a))l > PVax(a)lll - K^-max(a), ^Vax(a))l 

\ t-\ x\\\u Il2 Kill. II ATI / II "max (a) 111 ^ 1 1 P— max(fe) 1 1 1 

> (1 - 5)||/i max(a) || 2 - 6>||/i max(a) || 2 • V2k - a ( ^ h 2k -a 

^ n X l 2k ~ a a\\\U 112 flllL || 2 ||/3-max(fc)l|l 

> ^1 — — Y Pj||'imax(a)ll2 ~ y ll"-max(a) l|2 , • 
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Hence 



, H VT+6(e + r,) 

l A max(a)l|2 S - ,. . ==r "+ 



2 H/3-max(fc)||l 



1-5- ^/(2k-a)/a9 1-5- ^(2k-a)/a6 VW 
Applying Lemma 15, 5 1 with p = 2 and A = 2||/i_ max (£)||i yields 



(42) 



^ i=l i=k+l 



C?< 



\ 



£« + (. 



8=1 



^ + 



max(fc) || 1 so 



Vk 



-y 



< 






max(fc) 111 



\/fc 



< 



2ft 



< V2(l + J)fc/ a (e + ry) / 
1 - 5 - ^(2^ - a)/afi» \ 

Finally, it follows from Lemma 15.41 that 



^/2k/^e 



■ max(fc) 111 



y/k 



2 2 

1-5- v / (2k-a)/a9 \/2k - a Vk 



■ max(fc) ||l- 



'a,2k—a 



< 



2k -a 



min{6, 2k — a} 



,,2k-a} < max 



2k -a 



'-A>0a, 



a 



~L/a,b,k"a,b- 



o n \\U\\ <r y/2(l+5)k/a(e+r 1 ) ,, „ ,, 

5 ° l|ft|12 S l-<3"-C a , b , fc e + ^HP-max(fc)||l ^(l_5-C a , 6 , fe e)(2fc-a) t" Tfc 

of Theorem [221 



6 ''J "*'" y 2k-a 

-| — -= I j which finishes the proof 



The proof of Theorem 12.71 is basically the same, where we only need to use the inequalities 
\\A T Ah\\oo < \\A T (A/3 - ^lU + \\A T (y - AftWoo < (e + r,) and 

\(Ah, Ah max{a) )\ = \h^ (a) A T Ah\ < || Vax(a)l|iP T ^l|oo < y/a\\ h m3ux(a) \\ 2 (e + rj) 

instead of dMD and (HID. □ 



Proof of Theorem 12.41 and 12. 8L Again, it suffices to prove Theorem 12.81 We first prove 



2k 



the signal case. Set hi = diag(— =,••■ , — ,0, • • • ,0) G W. Since \\hi\\2 = 1, we can ex- 



/2k V2k 

tend h\ into an orthonormal basis {hi,--- ,h p } of W. Define the linear map A : 



by Ax = \ 2-a/(2k) ^i=2 c i^i f° r an x = Sf=i c *^«- The Cauchy-Schwarz Inequality yields 
that \(x, hi}\ < \\hi ■ lguppf^lbll^lb < V^O 2 '!! 2 ^ or a ^ a_s P arse vector x. Note that ||A£|| 2 = 



1 



a/(2fc) ^ Ibll 2 - ILlxll 2 - ^ ■ a/{2k) ^~ 112 ~- * A - a/{2k) 
2-a/{2k)J " " 2 ~ " " 2 " 



2/ (2k) J 



\x\\o and <5„ < 



2-a/(2k)J """^ a " 2-a/(2fc)' 

Now we estimate 0^ b . For any a-sparse vector xi and 6-sparse vector x 2 G K p with disjoint 
supports, write £1 = Ya=i d^i and X2 = Ym=i d ihi, we have 2 -a/(2k) Yn=i c ^ = ( x i> x %) = °- 
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1. When b < 2k — a, The Cauchy-Schwarz Inequality yields that \c\\ = \{h\,x\)\ < \/wt\\xi\\2 



and |di| = \(h 1 ,x 2 }\ < J&Wxih- So 



2* I 

_,U. IL Q^ 
2fel 



2-0/(2*;).. . „ .. ,A ,, 

i-i_i|(Asi, Ax 2 )| = I 2^cid»| = I - cidi| < 



2 IN ' " '^ ' ' ' - 2k 

j=2 

and consequently O: 6 < 2 _ a /( 2fc ) ' 2F- Hence 



a& 

112X22 



, _ nA ^ a/(2k) \2k-a 2k - a \ 2 Vab ^ 1 

* > + C -^^ 2^a/(4) + ^ (^T ' V —j • 2^7M^- L 

2. When b > 2k — a, if x\ = or x 2 = 0, it is clear that {Ax\,Ax 2 ) = < C 1 1 a?i 1 1 2 1 1 ^^2 1 1 2 f° r 
any C > 0. Without loss of generality, we assume that x\ and x 2 are non-zero and are 
normalized so that ||xi|| 2 = ||x 2 || 2 = 1. Since x\ and x 2 are a, 6-sparse respectively and 
x\ and X2 have disjoint supports, it follows from the Cauchy-Schwarz Inequality that for 
all A > 0, |ci| = |(/ii,xi)| < v^l^ilb = v^ and 



a / a / a I 2k 

id^xhl — T c il = \(hi,X2±\hn: — -^i>l < IN2 ± A /— r; — -xi||2 



t 2k — a \l 2k — a \l 2k — a ' \ 2k — a 

Hence, 

2 - a /(2k) 



-\(Axi,Ax 2 )\ = ly^Cjdil = | - cidi 



2 

i=2 



= |C1 " • ( maX{|dl + ^i^ Cl1 ' |dl " ^i^ Cl|} " '/^ Cl1 ) 

/ / 2k I a . A / a / Pk A 2 

< c 1 -[J- J- ci \=-J- W _ U + 



2k - a \ 2k - a I y 2k - a \\ 2a I 2Ja(2k 



y / a(2k — a) 
~ 2k 



where the last inequality follows from the facts that |ci| < yja/(2k) and a < k. So 

U a,b ^ 2-a/(2fc) 2fc anCl 



cA _ „ A ^ o/(2Ar) J 2fc - a /2fc-a| 2 */a(2A; ~ a ) ^ 1 

^ + ^<^2^^ + maX (^r'V^^}- 2-a/(2 fc ) 2. ^ 

To sum up, we have shown 8^ + C a ^^0^ h < 1. Furthermore, let 

n = (T~ ^~T, 0, • • • ) and v = (0~^~0, ^T~^~^T , 0, • • • ), 
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so u and v are both /c-sparse and Au = Av, since A(u — v) = 0. Suppose y = Ax\ = Ax2, then 
the fc-sparse signals u and v are not distinguishable based on (y,A). Finally, 8^ + C a fi,k^ab < •'■ 
is impossible by Theorem 12.51 so we must have 8^ + C a ^^Q^ b = 1. 

For the matrix case, the proof is essentially the same as the signal case. First we present 
the following lemma which can be regarded as an extension of the Cauchy-Schwarz Inequality 
(B,X) < \\B\\f\\X\\f with a constraint on rank(S). 

Lemma 5.6. Let X E R mxn (m < n) be a matrix with singular values \\ > A2 > • • • > A m; then 
for all B G ]j mxri with rank at most r, 



\{B,X)\<\\B\\ F 



N 



£*? 



2 



Then the matrix case can be proved by replacing the notations of vectors in the above proof 
by matrices and by using Lemma [5.61 instead of the Cauchy-Schwarz 's Inequality in the proof of 
the signal case. □ 

Proof of Lemma 13.11 For /c-sparse vectors /3, 7 £ W with disjoint supports, we can write 
them as j3 = X^jgTi a * e * an d 7 = SieT 2 ^* e * wnere Oj > 0, bj > 0, Ti is the support of /3, T2 is the 
support of 7, and ej is the vector with ith entry equals to ±1 and all others entries equal to zero. 
Correspondingly, suppose 1,7 6 ^mxn w ^k ran k a t most r, which satisfies X Y = XY = 0. 
Lemma 15.21 shows that they have singular value decompositions X = Yli&T a i u i v T an d Y = 
SieT biUivf, where the disjoint subsets T\ and T2 satisfy |Xi|, IT2I < r. We now consider the 
even and odd cases separately. 

Case 1. k, r > 2 is even. We focus on the matrix case. The proof of the signal case is similar. 
Without loss of generality, suppose X and Y are normalized so ||-X"||jr = ||V||.f = 1- Divide T\ 
and T2 into two parts, T\ = T\\ UT12, T2 = T21 UT22, such that Tn, T12, T21, T22 are disjoint and 
\Tij\ < r/2 for i,j E {1, 2}. Denote Xi = £) ieTl . <Wuf and Yi = J2ieT 2i b i u i v I '» * = 1, 2. Then 

2 
4 



2 2 



t,J=l 


«J=i 




< l t 


(1+0 x; «?-(!- 

ieT i:j L)T i: j 





^(||xi| 2 P + ||y|| 2 F 



2,5^ 
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and consequently O^}. < 25^ ■ Now in the example provided in the proof of Theorem 12.41 if 
a = b = k, we have 5 A = 1/3, 9yl = 2/3, which means the coefficient "2" in the inequalities of 
the even case in (|24p cannot be improved. 



Case 2. k,r > 3 is odd. For the proof of f|23|) and (|24p . we only show the matrix case as 
the signal case is similar. Since we can set a^ = or bi = for i ^ Ti or i ^ T2, Without loss 
of generality, we assume that |Ti| = r, | T2 1 = r, ai, bi might be for i G T\ U T2. Also without 
loss of generality, we can assume X and Y are normalized so ||-X'|||' = SieT a l = \ fTT an d 

||y||2 _ y^ l2 _ /r+T Tl^p^ 

II 1 Hf — 2^ieT 2 °i ~ y r-1- inen 



V-~i)/X-~s)/s l(MWlA<(,r)) 



r- 1 



(r-l)/2yV(^"3)/2 



r-1 



(M(%2 aiUivf),M(Y^ huivf)) 



i&Ti 



ieT 2 



£ 



ACTi,\A\=(r+l)/2, 

BCT 2 ,|B| = (r-l)/2 



\\M(%2 amvf + ^k 



(/,;('; 



.Tmi2 



ii-^E 



«6A 



ie-B 



JGA 



JS-B 



< Yl ((l+O-a-O) 



ACTi,|A|=(r+l)/2, 

SCT 2l |B| = (r-l)/2 



ieA ieB 



2r5 



vVI 



l)/2 J ((r - l)/2j S ^ + {(r - 3)/2j {(r + l)/2 J ^ 6 * 



25C 



M 



(r 
r- 1 



r- 1 



5 r ^ 



(r-l)/2/ V(r-3)/2 
r — 1 \ / r — 1 



' V^ 2 



£« 



(r-l)/2 ^ * (r + l)/2 



r — 1 



r — 1 



2r 



=^||x||p||y 



> - l)/2; V(r - 3)/2y v 7 ^^! V(^ - l)/2/ \(r - 3)/2J V^T 
which implies 8% < -^fcf^- 

Next we will construct an example for the signal recovery in the odd case where 6 A k 
A 5 A 7^ 0. Suppose k > 3 is odd and 2/c < p, denote 



Fll-r LF 




V2F ' ' ' ' ' ' " Z V2fc 

Similarly as in the proof of Theorem 12.41 we can extend j3\ and fii to an orthonormal basis of 
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W as {/3i, fo, ■■■ , P P }. Then for < A < 1, we define A : W -> W by 

p 
A(3 = Vl + AaiA + vT - Aa 2 /3 2 + E a *& 

i=3 

for /3 = ELi a ift- Jt is clear that for a11 Z 3 e M p , (1 - A)||/3||l < \\Ap\\% < (1 + A)||/3|||. Let /3 
and 7 be /c-sparse vectors with disjoint supports and ||/3||2 = II7II2 = 1- Then 

1(^/3,^7)1 = ^|P(/3 + 7)ll2-P(/3-7)ll2| 

< max {l^||/3 + 7 ||i- ^||/3 - 7 i, ^ll/? "Ti-^ll/^ + Ti 

= y(Pll^l7l^) = A||/3|| 2 ||7ll2 
which implies 9 kk < A. It can be easily verified that when 



/3 = (1,1--- ,1,0,---) and 7 =(0,0,--- ,0,1,1,--- ,1,0,---), 
we have |(.A/3, Ay)| = A||/3||2||7||2- These together imply 9 kk = A. 

Denote /3(i) as the ith entry of /3. Now let us estimate 5 k . For all fc-sparse /3 G W, suppose 

/3 = YJi=l c il3ii tnen 

||40||jj = (1 + A)!^,^)! 2 + (1 - A)|(/3,/3 2 )| 2 + E K/5, A>l 2 = II/3H3 + ACK/5,/301 2 - K/?,/? 2 >| 2 ) 

2 = 3 

2A; k 2k k Ik 

= wwl+Kif^mf-CZm- E /3(^)) 2 )/2fc = n/3in + A A( ^ /?(i))( ^ m . 

i=l j=l i=fc+l i=l i=j+l 

Suppose Ti = supp(/3)n{l,--- ,fe} and T 2 = supp(/3) n{/c + 1, • • • ,2fc}, then |Ti| + |T 2 | < k and 

KjXox E 0W)i = i(E /*«)(£ #o)i < Ji T ii E ^) 2 • i^i E ^) 2 

i=l j=fc+l ieTi i&T 2 y iSTi «eT 2 

£ MS E ^< v|r ' l( *- |Tll) iwi<^^iwi, 

JSTiUT 2 

where the last inequality is due to the facts that |Ti| is a non- negative integer and k is odd. It 
then follows that for all A;-sparse vector /3 S M. p , 

(1 - ^^A)||/3|| 2 < P/3|| 2 < (1 + ^^A)||/3|| 2 . 



2k Jur uz ~ " ' uz ~ v 2k 

It can also be easily verified that the equality above can be achieved for 

(fc+l)/2 (fc-l)/2 (fc-l)/2 

a = (i~^~i, o~^~o, i~^~i, 0, • • • ) 
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Hence 5^ = A \k • ^ n summar y; #& & = 7p ffir i n our setting, which implies that the constant 



2/v 



5= in (|23j) is not improvable. □ 



7k 
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